Rzszar-oh Article, 



Health Literacy Instruction and Evaluation 
among Secondary School Students 


Betty Hubbard and Jacquie Rainey 


ABSTRACT 


Background: Tobacco use, poor eating habits, and physical inactivity are the modifiable risk behaviors most associated 
with the leading causes of morbidity and mortality in the United States. Because these risk behaviors are established 
during adolescence, the nation’s schools are uniquely positioned to develop health literacy in students. Purpose: The 
purposes of this study were (1 ) to evaluate textbook-based health literacy instruction on the acquisition of health 
concepts and skills, and (2) to use the Health Education Assessment Project (HEAP) database items to assess changes 
in the concepts and skills associated with risk behaviors. Methods: The study participants consisted of 669 secondary 
students. Items were selected from the database to measure students’ achievement in concepts and skills related to 
tobacco, nutrition, and physical activity. Results: Results indicated significant improvements in concepts and skills 
scores from pretest to post-test in the treatment group when compared to the control group. Discussion: Exposure to 
health literacy instruction through a comprehensive, health education textbook has a positive influence on the devel- 
opment of concepts and skills that contribute to health literacy. Translation to Health Education Practice: Teachers 
should provide instruction from curricula that focus on health concepts and skills. Additionally, HEAP database items 
should be used for evaluative purposes. 


BACKGROUND 

Health education is recognized as an 
essential component of health promotion 
and a key to the reduction of risk behaviors 
associated with chronic diseases. 1 Health 
education is defined as “any combination 
of planned learning experiences based on 
sound theories that provide individuals, 
groups, and communities the opportunity 
to acquire information and the skills needed 
to make quality health decisions.” 2(p99> De- 
termining the influence of health education 
on the acquisition of this information (con- 
cepts) and these skills allows researchers to 
evaluate the efficacy of school-based instruc- 
tion as a tool for the achievement of health 
literacy and, therefore, as a component of 


chronic disease prevention. 

The nation’s schools are uniquely po- 
sitioned to educate children and youths 
about risk behaviors by making health 
education available to their students. The 
primary goal of school health education 
is to develop health literacy in the school- 
aged child. Health literacy is defined by 
the Joint Committee on Health Education 
Standards as “the capacity of individuals 
to obtain, interpret, and understand basic 
health information and services and the 
competence to use such information and 
services in ways which enhance health.” 3(p5) 
In 2005, a report by the National Academy 
of Sciences’ Institute of Medicine stated, 
“[T]he most effective means to improve 


health literacy is to ensure that education 
about health is a part of the curriculum at 
all levels of education.” 4(pl49) 

If the goal of school health education is to 
improve students’ health literacy, it is vital to 
have a framework to guide the achievement 
of this goal. In 1995, such a framework was 
developed and disseminated to states, school 
districts, administrators, and teachers across 
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the nation. Titled “National Health Educa- 
tion Standards: Achieving Health Literacy” 
(NHES) , the framework was developed by a 
coalition of health education organizations 
and professionals. The standards provided 
in this framework were designed to support 
schools in meeting the essential goal of en- 
abling students to acquire the concepts and 
skills necessary to achieve health literacy. Of 
the seven standards delineated in the frame- 
work, one (standard 1) addressed concepts 
and six (standards 2-7) addressed skills. 3 

Over the next decade, most states and 
many school districts across the country 
adopted these standards. Textbooks and 
other curricular materials were developed 
based on them, and the Health Education 
Assessment Project (HEAP) provided as- 
sessment items to evaluate the effectiveness 
of the curricular materials. 5 Due to research- 
based advances related to health education, 
the national standards were revised in 2006. 
The revised document, “National Health 
Education Standards: Achieving Excellence,” 
includes eight standards. Of these standards, 
one delineates concepts (standard 1) and 
seven (standards 2-8) delineate skills. 6 (Note: 
The revised standards were not available at 
the time this study was conducted. The 1995 
standards served as the foundation for the 
textbook series and the HEAP assessment 
items used in this study.) 

The emphasis on skills in the standards 
documents underscores the paradigm shift 
from merely informing youths about health 
concepts to also providing them with the 
skills that can enable them to make healthier 
decisions regarding their behaviors. 2,3,6,7 Cur- 
rent state-of-the-art health education cur- 
ricula provide sequenced instruction that is 
consistent with state and/or national health 
education standards. 3 These curricula are 
more likely to positively affect the health 
behaviors of students when they teach the 
skills needed to engage in health-enhancing 
behaviors. 8,9 

The shift from a cognitive-based to a 
skill-based philosophy has resulted in effec- 
tive programs in the areas of substance abuse 
prevention as well as sexually transmitted 
infections and pregnancy prevention among 


teens. Although research since the mid 
1980s has demonstrated the effectiveness of 
skill-based programs, these programs have 
overwhelmingly addressed a single issue 
(i.e., teen sexuality, violence, or tobacco/ 
substance use) and/or targeted specific 
populations (i.e., high-risk youth or mi- 
nority populations). 10 ' 16 A few programs — 
Growing Healthy (for grades K through 6), 
The Great Body Shop (for grades K through 
middle school), the Michigan Model (for 
grades K through 12), and the Teenage 
Health Teaching Modules (for grades 6 
through 12) — are comprehensive curricula 
that have demonstrated positive effects on 
students’ concepts and skills. However, these 
curricula are limited in their dissemination 
due to various factors such as the cost of 
the curricula, the inability of the developers 
to market widely, 17 ' 20 or exclusion from the 
state textbook adoption lists. 

PURPOSE 

Most health education in the United 
States is taught based on textbooks selected 
from state textbook adoption lists. To date, 
no comprehensive “state approved” textbook 
series has been researched to determine its 
efficacy in improving the health-related 
concepts and skills described in the NHES. 
Thus, the primary purpose of this study was 
to evaluate the effects of textbook-based 
health literacy instruction on students’ 
health-related concepts and skills. A second- 
ary purpose was to use HEAP assessment 
items to assess changes in concepts and skills 
related to the risk behaviors of tobacco use, 
poor nutrition, and physical inactivity. 

The resultant findings are important in 
determining whether health education, as it 
is most commonly delivered, influences the 
health literacy of children and youths. That 
is, does it develop the concepts and skills 
necessary for students to choose health- en- 
hancing behaviors that may prevent chronic 
diseases? The answer to this question can in- 
form persons who make decisions related to 
( 1 ) requiring health education in the nation’s 
schools, (2) determining the importance of 
this subject area, and (3) using HEAP assess- 
ment items to monitor the effectiveness of 


health education curricula. 

METHODS 

Design 

A quasi-experimental design was se- 
lected to control for the potential threats 
of maturation and history to the validity 
of the evaluation results. Intact classrooms 
were used to provide as natural a setting as 
possible for the instruction to occur. The 
goal was to test the efficacy of the instruction 
as it would occur in the regular classroom 
without experimental manipulation that was 
not replicable in the “real” school setting. 

Subjects 

Three schools from the Little Rock Pub- 
lic School District in Arkansas were selected 
for participation in the study. Two of the 
institutions are middle schools and one is 
a high school. These schools were selected 
based on adequate numbers of students in 
each health class and teacher access to a 
classroom. (Some health classes in Arkansas 
and elsewhere are taught in the gymnasium 
while several physical education classes are 
in progress.) An additional consideration 
was the availability of a control group. In 
Arkansas, students are required to take at 
least nine weeks of health and nine weeks 
of physical education during middle school. 
High school students must take one semes- 
ter of health and one semester of physical 
education. It is common practice to rotate 
classes of students from health to physical 
education and vice versa in order to fulfill 
the health and physical education require- 
ments during the school year. Students 
are randomly assigned to either health 
or physical education class via computer 
at the beginning of the school year. The 
control group consisted of students who 
were assigned to participate in physical 
education classes during the nine weeks or 
semester the study was conducted, rotating 
to health during the following nine weeks 
or semester. Students in the control group 
were not exposed to textbook-based health 
literacy instruction. 

All pretests for the health and control 
group classes were administered the first 
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week of school during the fall semester 
of 2005. Post-tests were administered to 
both groups prior to the end of the fall se- 
mester. Before data collection, a university 
Institutional Review Board (IRB) approved 
the study. 

Instruments 

An instrument consisting of 30 selected 
response items was developed for pre- and 
post-test assessment of health-related con- 
cepts and skills. The items were selected 
from the Council of Chief State School 
Officers (CCSSO) State Collaborative on 
Assessment and Student Standards (SCASS) 
HEAP database. The purpose of HEAP is to 
develop valid health education evaluation 
resources and to increase “capacity to align 
curriculum, instruction, and assessment 
to improve student health literacy through 
improved health instruction.” 21 The HEAP 
database was selected because the items have 
been rigorously developed, pilot tested, and 
determined to meet the requirements of 
content validity and reliability. 22 

Of the 30 items that were selected from 
the database, 15 assessed health concepts 
and 15 assessed health skills. Five health 
concept items queried students about to- 
bacco, physical activity, and nutrition. The 
15 selected skill items assessed students’ 
ability to demonstrate processes associated 
with accessing information, interpersonal 
communication, decision-making, goal 
setting, and self-management within the 
context of the topic areas (tobacco, nutrition, 
and physical activity). (Note: Because the as- 
sessment database is proprietary, the actual 
items are not included in this article.) 

The selected items were based on the 
lessons that were used in both the middle 
and high school textbooks. The reliability 
coefficient of the data was alpha=.75 and 
.77 for the middle school and high school 
skills scale, respectively, and .76 and .72 for 
the concepts scales, indicating an acceptable 
level of internal consistency. 

PROCEDURES 

A partnership was formed between the 
Arkansas Department of Education (Of- 
fice of Comprehensive School Health), the 


University of Central Arkansas (Depart- 
ment of Health Sciences), and the Little 
Rock Public School District. The latter was 
recruited as a partner because it had adopted 
a comprehensive textbook series for second- 
ary students during the previous academic 
year. Teen Health is the title of the middle 
school text and Glencoe Health is the high 
school text. This textbook series is published 
by Glencoe/McGraw-Hill and is listed on 
the Arkansas State Recommended List of 
Instructional Materials. The concepts and 
skills delineated in the NHES serve as the 
foundation for this series. 23,24 

Prior to pretest administration in the 
treatment and control group classes, a pas- 
sive consent form was distributed to each 
student. All persons who administered 
pretests and post-tests followed a written 
protocol to ensure conformity in adminis- 
tration of pre- and post-tests. 

Data Analysis 

Data was analyzed using SPSS 13 with a 
two-way ANOVA with repeated measures on 
the last factor. Reported p-values are based 
on two-sided tests with a p-value less than 
.01 considered significant due to multiple 
analyses. For the purposes of this study, 
the three topic areas (tobacco, nutrition, 
and physical activity) were combined into 
concept scores and skill scores. 

RESULTS 

Participants in pretesting included 377 
middle school students. Of these, 188 were 
in the treatment group, and 189 were in 
the control group. The high school pretest 
groups included 554 students. The treat- 
ment group consisted of 272 students, the 
control group 282. In the post-testing phase, 
208 middle school students participated 
(100 treatment, 108 control), along with 
461 high school students (230 treatment, 
231 control). Pre- and post-test scores were 
matched for each student. Demographics of 
the participants are presented in Table 1 

Differences in the Middle School Group 

Four mixed between-within subjects 
analysis of variances were conducted on the 
middle school and the high school scores of 
concepts and skills. The between subjects 


factor was group (treatment vs. control), and 
the within subject factor was time (pretest 
vs. post-test). Results are presented for the 
middle school students and then the high 
school students. There was a significant 
interaction effect on the concepts scores for 
treatment group by time [F(l,206)=120, 
pC.OOl], As can be seen in Table 2, the 
concepts scores improved in both the treat- 
ment and control groups, but the treatment 
group’s scores improved significantly more 
than the control group’s. The magnitude of 
the differences in the means was large (eta 
squared=.37). This result indicates that the 
amount the treatment group improved 
their scores from pre- to post-test (almost 5 
points) was significantly larger than the im- 
provement in the control group (less than 1 
point). Overall, 37% of the variability in the 
improvement from pre- to post-test scores 
can be attributed to the group the student 
was in, treatment versus control. 

Like the concepts scores, there was a 
significant interaction in the skills scores by 
treatment group and time [F( 1,206)= 123, 
p<.001, eta squared= .37], This result in- 
dicates that the treatment group students 
significantly improved their skills scores (by 
almost 5 points) compared to the control 
group (less than 1 point). The magnitude 
of this difference was considered large (eta 
squared=.37). Thirty-seven percent of the 
variability in the change in skills scores can 
be attributed to the treatment (Table 2). 

Differences in the High School Group 

Among the high school group, there 
was a significant interaction effect on the 
concepts scores for treatment group by 
time [F(l,459)=63, p<.001]. As can be seen 
in Table 2, the concepts scores improved in 
the treatment but not the control group. 
The magnitude of the differences in the 
means was between moderate and large 
(eta squared=.12). This result indicates that 
the amount the treatment group students 
improved their scores from pre- to post-test 
(almost 2 points) was significantly better 
than the slight decline in the control group’s 
scores. Overall, 12% of the variability in the 
change from pre- to post-test scores can be 
attributed to the health instruction. 
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Table 1 

Participant Demographics by Group and Grade (n=669) 





Treatment 



Control 



Middle School 

High School 

Middle School 

High School 


n 

% 

n 

% 

n 

% 

n 

% 

Sex 









Missing 

1 

1 .0% 

1 

.4% 

1 

.9% 

0 

.0% 

Female 

55 

55.0% 

115 

50.0% 

50 

46.3% 

1 13 

48.9% 

Male 

44 

44.0% 

1 14 

49.6% 

57 

52.8% 

1 18 

51.1% 

Total 

100 


230 


108 


231 


Race 









White 

14 

14.4% 

77 

35.3% 

19 

19.0% 

88 

39.8% 

Black 

75 

77.3% 

120 

55.0% 

73 

73.0% 

1 13 

51.1% 

Hispanic 

2 

2.1% 

6 

2.8% 

3 

3.0% 

6 

2.7% 

Asian 

1 

1 .0% 

6 

2.8% 

0 

.0% 

10 

4.5% 

Biracial 

5 

5.2% 

9 

4.1% 

4 

4.0% 

0 

.0% 

Native American 

0 

.0% 

0 

.0% 

1 

1.0% 

4 

1 .8% 

Total 

97 


218 


100 


221 



There was a significant interaction in the 
skills scores by treatment group and time 
[F(l,459)=123, p<.001, eta squared=.21]. 
This finding indicates that the treatment 
group students significantly improved their 
skills scores (by more than 2.6 points) com- 
pared to the control group (less than one- 
fourth of a point). Twenty-one percent of the 
variability in the change in skills scores can 
be attributed to the treatment (Table 2). 

DISCUSSION 

The primary purpose of this study was 
to evaluate the influence of comprehensive 
textbook-based instruction on students’ 
acquisition of health-related concepts and 
skills leading to health literacy. Data from 
this study suggests that exposure to such 
instruction resulted in a significant improve- 
ment in the concepts and skills described 
in the NHES. A secondary purpose of the 
study was to use HEAP assessment items to 
evaluate changes in concepts and skills as- 
sociated with tobacco, nutrition, and physi- 
cal activity, leading risk factors in chronic 
diseases. The HEAP assessment items proved 
to be a valid and reliable form of evaluating 
concept and skill acquisition. Study results 
indicate that students who were exposed to 


the textbook-based instruction achieved 
significant increases in their scores on the 
HEAP assessment items. 

These findings are consistent with previ- 
ous research that indicates the effectiveness 
of health education aligned with the NHES. 
That is, although state-of-the-art health 
education instructs students about essential 
concepts, it focuses on teaching the skills 
needed to adopt healthy behaviors. Previ- 
ous research has shown that curricula that 
transmit accurate information without in- 
corporating skill development are less likely 
to influence health behavior. 8,9 

If exposure to comprehensive textbook- 
based instruction improves the health lit- 
eracy of adolescents, the inclusion of such 
instruction is to be encouraged in all schools. 
Unfortunately, national data indicates that 
students receive little and inconsistent health 
education instruction. The Schools Health 
Policy and Promotion Study (SHPPS) 
conducted in 2000 demonstrated that the 
percentage of elementary schools requiring 
health education in grades K-5 ranged from 
33 to 45%. Only 27% of schools require 
health education in grade 6, 20% in grade 8, 
10% in grade 9, and 2% in grade 12. 25 Given 
the potential for health education to impact 


health literacy, it appears that the nation’s 
schools are missing a prime opportunity to 
influence the health of its citizens. 

Ideally, health education would be in- 
corporated as a part of a coordinated school 
health program within each school. This 
program would consist of eight interactive 
components that work together to enhance 
the health status of students, staff, and com- 
munities. In addition to health education, 
these components would include physical 
education; health services; counseling, 
psychological, and social services; nutrition 
services; healthy school environment; par- 
ent, family, and community involvement; 
and health promotion for school faculty and 
staff. The effectiveness of health education is 
increased when its outcomes are reinforced 
by the other seven components. 26 It should 
be noted that three of the components — 
health education, physical education, and 
nutrition services — are directly related to 
the behaviors associated with increased risk 
of chronic diseases. 

The coordinated school health program 
may be viewed as an essential part of health 
promotion within each community. Health- 
enhancing policies such as smoking bans 
would reinforce the no-smoking messages 
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Table 2. Concepts and Skills Pretest and 
Post-test Scores by Group and Grade 




Treatment 



Control 



Mean 

Standard 

Deviation 

n 

Mean 

Standard 

Deviation 

n 

Middle School 

Concepts pretest 

6.47 

2.44 

100 

5.89 

2.43 

108 

Concepts post-test 

11.19 

2.63 

100 

6.77 

2.50 

108 

Skills pretest 

6.74 

2.22 

100 

5.69 

2.25 

108 

Skills post-test 

1 1.72 

2.61 

100 

6.64 

2.08 

108 

High School 

Concepts pretest 

8.42 

2.87 

230 

8.20 

2.75 

231 

Concepts post-test 

10.12 

2.91 

230 

8.16 

3.05 

231 

Skills pretest 

8.38 

2.70 

230 

7.89 

2.82 

231 

Skills post-test 

1 1.04 

2.94 

230 

8.12 

3.08 

231 

Note: The possible range of test scores was 0-15. 






received at school. These policies and mes- 
sages would, in turn, be reinforced by anti- 
smoking messages in the media and the high 
tax rates placed on tobacco products. 

In conclusion, the results of this study 
demonstrate that comprehensive health 
education instruction using standards-based 
texts plays an important role in the develop- 
ment of health literacy and possible reduc- 
tion of risk factors that contribute to chronic 
diseases. Therefore, the potential power of 
health education to influence health behav- 
iors and, ultimately health status and health 
care spending cannot be ignored. 

Strengths and Limitations 

The strength of this study was the au- 
thentic setting in which the research was 
conducted. Participating teachers received 
no in-depth, in-service training prior to 
teaching health education. They were in- 
structed to teach the content areas related 
to tobacco, nutrition, and physical activity. 
However, consistent with the setting, teach- 
ers made their own decisions about how to 
present the chapters in the text as well as how 
to use the ancillary materials provided as 
part of the textbook series. The texts were se- 
lected by teachers in the school district using 
the State Recommended List of Instructional 


Materials, the most common method of 
curriculum selection across the nation. 

The primary limitation of the study was 
the number of students who participated in 
pretesting but were unavailable for post- 
testing. Only those participants with a 
matching pretest and post-test were used 
in the analyses. Therefore, two hundred 
sixty-two students ( 28 %) who completed 
pretests were lost from the sample. Many 
factors such as relocation, absence at the 
time of the post-test, and incomplete data 
contributed to the reduction in sample 
size. However, the study attrition rate is 
consistent with district-wide attrition 
rates. Second, because this study was con- 
ducted with a sample of urban students 
in a public school district, results may 
not be generalized nationally or within 
different school environments. Third, the 
majority of the sample consisted of White 
and Black students. A small percentage 
of the participants identified themselves 
as Hispanic, Asian, Native American, and 
biracial. A final limitation was the lack of 
a double blind research design. Instructors 
and students in both the experimental and 
control groups were aware that they were 
participating in a research study. 


TRANSLATION TO HEALTH 
EDUCATION PRACTICE 

This study explored the effect of com- 
prehensive textbook-based instruction on 
students’ health concepts and skills. Based 
on the findings, a few recommendations 
can be made: 

• Providing time in the curriculum for 
health education should be a primary focus in 
order to increase the health literacy of children 
and youths. 

• Teachers and other personnel should 
select curricular materials that emphasize the 
concepts and skills delineated in the NHES. 

• Increased effort should be made to re- 
search the effects of comprehensive curricula. 
Curricula that are proven effective should be 
widely disseminated. 

• Future research should include adoles- 
cents of other racial/ethnic groups. 

• The HEAP database provides concept and 
skill items that can be used by educators for 
process and impact evaluation efforts. Edu- 
cators should use these items for curricular 
assessment and support the development of 
additional database items. 
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